Repository navigation
feat(router): bridge queue admission lifecycle - #13378
Closed
ishandhanani wants to merge 2 commits into
Closed
ishandhanani wants to merge 2 commits into
ishandhanani wants to merge 2 commits into
Conversation
|
ishandhanani
force-pushed
the
idhanani/dyn-3231-live-progress
branch
2 times, most recently
from
August 17, 2026 19:10
6f7f1c8 to
928e916
Compare
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/dyn-3231-live-progress
branch
from
August 17, 2026 21:48
928e916 to
edd3df9
Compare
ishandhanani
changed the base branch from
idhanani/dyn-3231-session-aware-policy-crate
to
idhanani/dyn-3231-queue-admission-host
August 17, 2026 21:48
Contributor
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This completes the queue-admission lifecycle by exposing terminal reservation events through Selection Service and carrying live response progress to the policy. It makes ThunderAgent’s session accounting observe dispatch, success, abort, and bounded output growth without changing the no-policy path.
How This Was Implemented
Walkthrough
Mental model
The host PR makes a policy decision and owns deferred requests. This PR closes that loop once a request crosses the router boundary: Selection Service reports terminal reservation state, and the response path optionally publishes monotonic output growth.
sequenceDiagram participant P as Admission policy participant Q as Queue actor participant S as Selection Service participant R as Response path P->>Q: Ready or Defer Q->>S: select and reserve S-->>Q: reservation and progress updater Q->>R: dispatch with request guard R->>P: progress at output-block boundary R->>Q: completed or aborted Q->>P: terminal lifecycle eventRequest lifecycle
A deferred reservation that is aborted before dispatch is removed from host-owned deferred storage, gets an
Abortedpolicy event, and completes the original selection with a typed terminal error. On dispatch or stream failure,RequestGuard::abort()is awaited before the error reaches the retry manager; itsDropcleanup only covers consumer disconnect and cannot start a migration retry.Progress readers and updaters are created only for policy-managed requests. The response path updates the reader monotonically at existing output-block boundaries, while terminal
Completedcontains the exact final input-plus-output context.Boundaries and limitations
This PR does not add a response-body callback to policies, change retry selection semantics, or alter regular requests that do not configure queue admission. Live progress is best-effort between dispatch and the terminal event, not a substitute for terminal context.
Validation
cargo test -p dynamo-kv-router --features standalone-selection --lib— 997 passed.cargo test -p dynamo-llm --lib kv_router::push_router::tests -- --nocapture— 12 passed.cargo check -p dynamo-llm --quiet.cargo clippy -p dynamo-kv-router --lib -- -D warnings.